Towards a Consistent Interpretation of AIOps Models

Artificial Intelligence for IT Operations (AIOps) has been adopted in organizations various tasks, including interpreting models to identify indicators of service failures. To avoid misleading practitioners, AIOps model interpretations should be consistent (i.e., different on the same task agree with one another feature importance). However, many studies violate established practices machine learning community when deriving interpretations, such as suboptimal performance, though impact violations interpretation consistency not studied. In this paper, we investigate along three dimensions: internal consistency, external and time consistency. We conduct a case study two tasks: predicting Google cluster job failures, Backblaze hard drive find that randomness from learners, hyperparameter tuning, data sampling controlled generate interpretations. AUCs greater than 0.75 yield more compared low-performing models. Finally, are constructed Sliding Window or Full History approaches have most trends presented entire datasets. Our provides valuable guidelines practitioners derive interpretation.